Machine learning and recognition is a field of study and applications whereby machines, in the form of dedicated hardware, computing software or combinations thereof, learn the key features and characteristics of objects. The objects may be physical in nature, examples of which are digitized images of automobiles or human faces. The objects may also be non-physical in nature, examples of which are sets of digital information representing the shopping information of customers. The characteristics of the objects are provided to the machine in the form of digital arrays of data that are known as feature vectors, or simply “vectors”. Individual elements of the feature vectors are known as components. The machine is capable of holding many such feature vectors, and may use one of many algorithms from the field of neural networks during learning to assign the feature vectors to a class of objects. The machine may be capable of holding and analyzing un-related sets of data, with the data sets known as a “context”. For example, it may contain a group, or context of feature vectors related to automobiles and another context containing feature vectors related to consumer spending habits. The Machine could direct new feature vectors requiring analysis to the appropriate context. A context may be further sub-divided into categories.
Once the machine has learned an appropriate number of features, the characteristics of new objects are given to the machine in the form of vectors for classification; that is, to determine if the new object belongs to a class of previously learned objects. The machine may use one or more algorithms from the field of neural networks to determine the closeness (conversely, distance) of a new feature vector to the learned feature vectors. The distance between a learned vector and a new observed vector is often performed using a form of Euclidian or Manhattan distance calculation and results aggregation. One example distance calculation is the Manhattan L1 norm distance, also known as the “taxi cab” distance. Another distance calculation is the Euclidian L2 norm. A third example is Lmax or L∞. A machine performing this analysis may be known as a classifier.
For machine learning to be increasingly practical in today's digital environment it needs to be conducive to various data widths and resolutions, support averaging and probabilistic calculations, as well as have the capability to swap in and out “files” (or classes of learned data) to support multiuser and/or multipurpose application scenarios. The machine may also be required to perform these tasks at very high rates of speed.
Hardware implementations of neural network algorithms saw significant interest in the 1980's but predominantly took the approach of weights in a multi-layer perceptron. Many of these solutions were analog in nature. Recent efforts in this space have rekindled an interest in analog and “spiking neurons” that try to conform very closely to biological brain cells. These approaches—using weights in a multi-layer perceptron and spiking neurons—are a different approach from that of the 1980s, may also be digital in nature, but are different than the Radial Basis Function (RBF) and Restricted Coulomb Energy (RCE) algorithms approaches. IBM subsequently patented and pursued early generations of a hardware implementation for the base RBF/RCE/kNN architecture. The more practical approach disclosed herein may be well suited for heterogeneous environments, or in some cases, standalone environments.
Disclosed herein are circuits and functions that will enhance an RBF/RCE/kNN based architecture. Their usefulness in a general computing environment performing digital memory based “fuzzy” operations in a hardware implementation offers significant performance improvements made by emulating the important computational attributes of neural networks without the issues of trying to emulate unnecessary biological functions. Software simulations that use von Neumann compatible data types and techniques may more readily transition to parallel memory based and scalable computational approaches with these RBF/RCE, kNN embodiments. Embodiments of multiple performance embodiments are also set forth herein.